BioMeT and algorithm challenges: A proposed digital standardized evaluation framework

Technology is advancing at an extraordinary rate. Continuous flows of novel data are being generated with the potential to revolutionize how we better identify, treat, manage, and prevent disease across therapeutic areas. However, lack of security of confidence in digital health technologies is hampering adoption, particularly for biometric monitoring technologies (BioMeTs) where frontline healthcare professionals are struggling to determine which BioMeTs are fit-for-purpose and in which context. Here, we discuss the challenges to adoption and offer pragmatic guidance regarding BioMeTs, cumulating in a proposed framework to advance their development and deployment in healthcare, health research, and health promotion. Furthermore, the framework proposes a process to establish an audit trail of BioMeTs (hardware and algorithms), to instill trust amongst multidisciplinary users

Verification, Analytical Validation, and Clinical Validation (V3): The Foundation of Determining Fit-for-Purpose for Biometric Monitoring Technologies (BioMeTs)

Digital medicine is an interdisciplinary field, drawing together stakeholders with expertize in engineering, manufacturing, clinical science, data science, biostatistics, regulatory science, ethics, patient advocacy, and healthcare policy, to name a few. Although this diversity is undoubtedly valuable, it can lead to confusion regarding terminology and best practices. There are many instances, as we detail in this paper, where a single term is used by different groups to mean different things, as well as cases where multiple terms are used to describe essentially the same concept. Our intent is to clarify core terminology and best practices for the evaluation of Biometric Monitoring Technologies (BioMeTs), without unnecessarily introducing new terms. We focus on the evaluation of BioMeTs as fit-for-purpose for use in clinical trials. However, our intent is for this framework to be instructional to all users of digital measurement tools, regardless of setting or intended use. We propose and describe a three-component framework intended to provide a foundational evaluation framework for BioMeTs. This framework includes (1) verification, (2) analytical validation, and (3) clinical validation. We aim for this common vocabulary to enable more effective communication and collaboration, generate a common and meaningful evidence base for BioMeTs, and improve the accessibility of the digital medicine field

Defining the Digital Measurement of Scratching During Sleep or Nocturnal Scratching: Review of the Literature

BackgroundDigital sensing solutions represent a convenient, objective, relatively inexpensive method that could be leveraged for assessing symptoms of various health conditions. Recent progress in the capabilities of digital sensing products has targeted the measurement of scratching during sleep, traditionally referred to as nocturnal scratching, in patients with atopic dermatitis or other skin conditions. Many solutions measuring nocturnal scratch have been developed; however, a lack of efforts toward standardization of the measure’s definition and contextualization of scratching during sleep hampers the ability to compare different technologies for this purpose. ObjectiveWe aimed to address this gap and bring forth unified measurement definitions for nocturnal scratch. MethodsWe performed a narrative literature review of definitions of scratching in patients with skin inflammation and a targeted literature review of sleep in the context of the period during which such scratching occurred. Both searches were limited to English language studies in humans. The extracted data were synthesized into themes based on study characteristics: scratch as a behavior, other characterization of the scratching movement, and measurement parameters for both scratch and sleep. We then developed ontologies for the digital measurement of sleep scratching. ResultsIn all, 29 studies defined inflammation-related scratching between 1996 and 2021. When cross-referenced with the results of search terms describing the sleep period, only 2 of these scratch-related papers also described sleep-related variables. From these search results, we developed an evidence-based and patient-centric definition of nocturnal scratch: an action of rhythmic and repetitive skin contact movement performed during a delimited time period of intended and actual sleep that is not restricted to any specific time of the day or night. Based on the measurement properties identified in the searches, we developed ontologies of relevant concepts that can be used as a starting point to develop standardized outcome measures of scratching during sleep in patients with inflammatory skin conditions. ConclusionsThis work is intended to serve as a foundation for the future development of unified and well-described digital health technologies measuring nocturnal scratching and should enable better communication and sharing of results between various stakeholders taking part in research in atopic dermatitis and other inflammatory skin conditions

Remote and interdisciplinary research in surgical knowledge production.

BACKGROUND: Surgical knowledge production has changed dramatically in the last 30 y, moving away from investigations by individual surgeon researchers and toward remote and interdisciplinary research. We investigated how surgeons make decisions about engaging in research and identify motivators, facilitators, and barriers to conducting research in an increasingly challenging environment. MATERIALS AND METHODS: We performed a qualitative analysis of semistructured interviews with surgeons from academic medical centers across the United States. We asked participants to describe their experiences and opinions regarding remote and interdisciplinary collaborations. RESULTS: Of 64 surgeon researchers invited, 21 (33%) agreed and participated in semistructured interviews. Each interview lasted an average (standard deviation) of 29 min (12). Surgeons were motivated by both internal and external factors, including some that might be identified as barriers. The internal desire to improve care and the need for collaboration to address increasingly complex questions requiring larger samples sizes emerged as most significant to interview participants. Social networks were identified as the dominant facilitator of multisite research, with technology playing a supporting role. Barriers to remote and interdisciplinary research ranged from individual, micro level barriers, through structural barriers that include institutional level challenges and competing priorities, to macrolevel system and policy-level barriers. CONCLUSIONS: Surgeons clearly recognize the importance of high-quality research aligned with current paradigms of clinical care and are using remote and interdisciplinary collaboration to improve the quality of the science they produce and align their work with the demand for increasingly high levels of evidence

A systematic review of feasibility studies promoting the use of mobile technologies in clinical research

Mobile technologies such as smartphone applications, wearables, ingestibles, and implantables, are increasingly used in clinical research to capture study endpoints. On behalf of the Clinical Trials Transformation Initiative, we aimed to conduct a systematic scoping review and compile a database summarizing pilot studies addressing mobile technology sensor performance, algorithm development, software performance, and/or operational feasibility, in order to provide a resource for guiding decisions about which technology is most suitable for a particular trial. Our systematic search identified 275 publications meeting inclusion criteria. From these papers, we extracted data including the medical condition, concept of interest captured by the mobile technology, outcomes captured by the digital measurement, and details regarding the sensors, algorithms, and study sample. Sixty-seven percent of the technologies identified were wearable sensors, with the remainder including tablets, smartphones, implanted sensors, and cameras. We noted substantial variability in terms of reporting completeness and terminology used. The data have been compiled into an online database maintained by the Clinical Trials Transformation Initiative that can be filtered and searched electronically, enabling a user to find information most relevant to their work. Our long-term goal is to maintain and update the online database, in order to promote standardization of methods and reporting, encourage collaboration, and avoid redundant studies, thereby contributing to the design and implementation of efficient, high quality trials

Sensor Data Integration: A New Cross-Industry Collaboration to Articulate Value, Define Needs, and Advance a Framework for Best Practices

Data integration, the processes by which data are aggregated, combined, and made available for use, has been key to the development and growth of many technological solutions. In health care, we are experiencing a revolution in the use of sensors to collect data on patient behaviors and experiences. Yet, the potential of this data to transform health outcomes is being held back. Deficits in standards, lexicons, data rights, permissioning, and security have been well documented, less so the cultural adoption of sensor data integration as a priority for large-scale deployment and impact on patient lives. The use and reuse of trustworthy data to make better and faster decisions across drug development and care delivery will require an understanding of all stakeholder needs and best practices to ensure these needs are met. The Digital Medicine Society is launching a new multistakeholder Sensor Data Integration Tour of Duty to address these challenges and more, providing a clear direction on how sensor data can fulfill its potential to enhance patient lives

Recent Academic Research on Clinically Relevant Digital Measures: Systematic Review

BackgroundDigital clinical measures collected via various digital sensing technologies such as smartphones, smartwatches, wearables, ingestibles, and implantables are increasingly used by individuals and clinicians to capture health outcomes or behavioral and physiological characteristics of individuals. Although academia is taking an active role in evaluating digital sensing products, academic contributions to advancing the safe, effective, ethical, and equitable use of digital clinical measures are poorly characterized. ObjectiveWe performed a systematic review to characterize the nature of academic research on digital clinical measures and to compare and contrast the types of sensors used and the sources of funding support for specific subareas of this research. MethodsWe conducted a PubMed search using a range of search terms to retrieve peer-reviewed articles reporting US-led academic research on digital clinical measures between January 2019 and February 2021. We screened each publication against specific inclusion and exclusion criteria. We then identified and categorized research studies based on the types of academic research, sensors used, and funding sources. Finally, we compared and contrasted the funding support for these specific subareas of research and sensor types. ResultsThe search retrieved 4240 articles of interest. Following the screening, 295 articles remained for data extraction and categorization. The top five research subareas included operations research (research analysis; n=225, 76%), analytical validation (n=173, 59%), usability and utility (data visualization; n=123, 42%), verification (n=93, 32%), and clinical validation (n=83, 28%). The three most underrepresented areas of research into digital clinical measures were ethics (n=0, 0%), security (n=1, 0.5%), and data rights and governance (n=1, 0.5%). Movement and activity trackers were the most commonly studied sensor type, and physiological (mechanical) sensors were the least frequently studied. We found that government agencies are providing the most funding for research on digital clinical measures (n=192, 65%), followed by independent foundations (n=109, 37%) and industries (n=56, 19%), with the remaining 12% (n=36) of these studies completely unfunded. ConclusionsSpecific subareas of academic research related to digital clinical measures are not keeping pace with the rapid expansion and adoption of digital sensing products. An integrated and coordinated effort is required across academia, academic partners, and academic funders to establish the field of digital clinical measures as an evidence-based field worthy of our trust